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ABSTRACT 

Side-developments of an instructional systems 
development team which are themselves important products of a 
research effort are documented in this report. The five studies 
presented include an investigation of the comparability of 
computer- assisted and conventional test admin istrationf prompting and 
confirmation as instructional strategies with computer-assisted 
instruction, analysis of student performance records, a survey of 
computer scinece literature relevant to computer- assisted 
instruction, and a computer-assisted approach to spelling. (EM 011 
038 through EM Oil 043, EM Oil 046, EM Oil 047, and EM Oil 049 
through EM Oil 058 are related documents) . (SH) 
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FOREWORD 



An active instructional systems development team will often take on a 
specific mission to fulfill the objectives of a grant. Enroute to that goal 
some members of the team inevitability follow serendipitous leads which result 
in exploration of many new and exciting ideas. This semi-annual report although 
delayed for valid reasons, provides documentation on some of these less planned, 
but nevertheless important products of a major research effort. 

In the management of the present grant from the U. S. Office of Education 
it has been our practice to encourage talented staff members to follow some new 
and interesting leads of their own choosing. We think the results of these 
explorations provide grist for the development of future proposals and should 
be shared with the community of scholars interested in computer applications to 
instruction. 



Harold E. Mitzel 
University Park, Pa. 
June 11 , 1971 
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COMPARABILITY OF COMPUTER-ASSISTED AND CONVENTIONAL 
TEST ADMINISTRATION 

Robert B King and William Rabinowitz 

Studies dealing with learner variables under computer-assisted instruction 
(CAI) frequently require the use of standardized performance measures. It may 
be desirable to precede CAI programs with standardized aptitude tests to select 
the appropriate level and type of instruction for each individual, and achieve- 
ment tests are almost always needed at the end of instructional sequences tr 
measure proficiency and diagnose learner difficulties. Although such tests 
may be administered by traditional methods, there are certain advantages in 
having the computer conduct the administration. Apart from the more obvious 
benefits of **apid scoring and diagnostic decision making, computerized test 
administration has a natural appeal in view of the computer's ability to 
control many troublesome test administration variables. Greater uniformity 
acoss administrations can be achieved by controling item exposure time and 
^♦eoi sequence, and recent evidence (Heckman, Tiffin, and Snow, 1967) indicates 
that item exposure control will increase internal consistency without affect- 
int other test characteristics adversely 

Caution Is warranted, however, in assuming that computer-administered 
test resuHs are comparable to those from conventional administration. 
Certain aspects of the computer testing environment such as the absence of an 
examiner, the presence of complicated interface equipment, and the Isolation 
of the student are unique to computerized administration and constitute 
potential contributors to test variance If these aspects produce sizable 
components of reliable variance, normative data based on conventional adninis- 
tration may be misleading, 

A number of researchers (Hopkins, Lefever, and Hopkins, 1967; Fargo, 
et al., 1967; Heckman, et al, , 1967; Curtis and Kropp, 1961a, 1961b; Burr, 1963) 
have investigated the comparability of printed versus televised or visual- 
slide-projected test administration Their results, in general, indicate no 
substantial differences among these presentation media in terms of item and 
test characteristics. Televised and slide -projected actaiini strati on are group 
modes, however, and use conventional printed response forms. Thus generaliza- 
tion cannot be made to computer administration where the student responds via 
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Machine In a relatively Isolated setting. Also, the above studies used 
achlevenent and Intelligence tests for comparison - neasures not particularly 
sensitive to variation In the test conditions. O'Bvyan and Boersna (1968) 
recently deaonstrated that test presentation variations that have little 
effect on verbal and nonverbal Intelligence scores can produce considerable 
differences In neasures such as creativity. Thus, If computer adnlnl strati on 
Introduces new sources of variance not foimd In conventional testing nodes, 
the effect would probably be nost likely to occur In measures of creativity. 
The purpose of the research reported here, therefore, was to compare conven- 
tional group administration and computer-assisted administration of a test of 
creativity. 



Method 

Two separate studies were conducted using different Ss and slightly 
different procedures. They used the same basic measures and analysis, how- 
ever, and thus m»y conveniently be presented together. Study I Mas a pre- 
liminary effort to the larger, more extensive Study II. 

Subjects 

Study I used 37 paid volunteer male technical education (two-year program) 
students enrolled at The Pennsylvania SUte University Random assignment was 
awde to either a computer-assisted or a conventional test administration 
condition. Stu^ II used 107 ( 36 male. 71 female) volunteer undergraduates 
enrolled In baccalaureate programs at The Pennsylvania SUte University. Ss 
within each sex were randomly assigned to the two test conditions to Insure 
equal proportions of males and females under each condition. 

Procedures and Eouloment 

The primary measuring Instrument selected for both studies was Form 1 of 
the Remote Associates Test (RAT. Nednick and Mednick, 1967). The RAT Is a 
verbal test of creativity consisting of 30 Items, each requiring a single 
verbal response. Each Item consists of three words and the Ss has to arHve 
at the appropHate response to these words through associative processes. An 



example Is the three words rat, blue, and cottage. The S is required to find 
a fourth word which serves as a remote associative link to each of these words. 
Cheese is the correct answer in this case. 

In the computer-assisted test condition, the RAT Form 1 was converted for 
computer use with the Coursewriter CAI language. Test instructions under the 
computer-assisted condition paralleled the standard printed form as nearly as 
possible. The 30 items of the RAT were placed on 2 x 2 inch slides, and 
presented via random access slide projectors operated in conjunction with four 
IBM 1050 typewriter terminals. All four terminals were connected to an IBM 
1410 computer located on the main campus. The computer version of the test 
was untlmed, and item progression was controlled by S. As each item was 
presented, Ss would either type an answer or choose to skip the item. No time 
limits were imposed for items, but response latencies were recorded by the 
computer, and Ss were urged to answer more rapidly or consider skipping items 
when latencies on proceeding items exceeded three minutes. All items skipped 
were recycled and presented in sequence a second, and if necessary, a third 
time. S receiving an unanswered item for the third time were instructed to 
guess at the item if they were still unable to decide upon an answer. Tests 
administered under this condition were scored on the basis of total number of 
items correct, and specific items missed were recorded on the print-out. All 
scoring was done by the computer which was programed to accept a variety of 
spelling options for each correct answer. 

Ss recieving the RAT Form 1 under the conventional administration 
condition were given the standard printed test forms in the presence of an 
examiner. Administration followed the procedures specified in the RAT 
Examiner's Manual , with the exception that the tests were wtimed instead of 
adhering to the 40-minute time limit recommended in the manual. Since only 
four students could be tested simultaneously under the computer-assisted 
condition, Ss assigned to the conventional condition were tested in small 
groups of approximately four each at times corresponding to the computer 
assisted testing session. Test administration for both groups took place over 
a five-di(y interval. 

All procedures involving the administration of RAT Form 1 under both 
conditions were identical for Studies I and II. Study II differed from Study 
I, in that after Ss in both administration conditions had completed RAT Form 1, 



they received four additional reference tests administered at one central 
location using conventional procedures. The four reference measures were the 
RAT Form 2, an unstandardized verbal fluency measure calleU Anagram Frequency 
by the investigators, a scoring variation of the Anagram Frequency measure 
called Anagram Originality, and an unstandardized paper and pencil opinion 
ticale called Flexibility Inventory. 

The first three reference measures were selected because they were easily 
administered paper and pencil measures known to correlate wi*h the RAT Form 1. 
T.ie RAT Fonii 2 is not quite as reliable as Form 1. but Mednick and Mednick 
(1967) report that the two forms correlate to the extent of .81. The two 
anagram measures were employed by Higgins (1966) in an investigation of cor- 
relates of the RAT Form 1. and are reported by Mednick and Mednick (1967) as 
part of the construct validity evidence supporting the RAT. Higgins asked Ss 
to construct as many words as possible, of four or more letters, from the word 
"ooneration." Scores were given for the total nunter of words and the ortgi- 
nality of words (defined as words given by no more than two other Ss.) The 
frequency measure correlated .31 with RAT scores, and the originality measure 
correlated .43. 

The fourth criterion measure was a forced-choice opinion inventory 
developed by the second author and believed to measure a dimension best 
described as "flexibility of thinking." It was employed in the present study 
to detect possible interactions between administration mode, creativity, and a 
potentially related attitudinal measure. 

Analysis 

The problem of determining whether computer-assisted testing and conven- 
tional testing are directly comparable is really a problem of determining the 
equivalence of two forms of the same test— one administered by computer and 
the other by conventional procedures. 

Medley (1957) has proposed a procedure for testing the equivalence of 
two tests which examines four criteria: equality of means, equality of vari- 
ances, equality of errors of measurement, and "homogeniety of function." The 
procedure is based upon a two-factor analysis of variance with repeated 
measures on one factor, and is especially appropriate for use in studies of 



the type reported here where carry-over effects necessitate the use of dif- 
ferent subjects for the two test forms (or 1n this case two modes of adminis- 
t'^ati on). 

In the oresent experiment, the Medley test for equivalence was applied 
^n both Study I and Study II to compare the performance cf the computer- 
assisted and conventional groups on ^'e RAT Form I The two administration 
mooes coupled with the 30 items of the RAT resulted in a 2 X X factorial de- 
sign* with repeated measures across subject. In the Medley procedure each of 
the four criteria for equivalence (equal means, equal variances, equal errors 
of measurement, and homogeniety of function) can be stated as a null hypo- 
thesis, and if any are rejected the tests In question cannot be regarded as 
equivalent If all four null hypothesis cannot be rejected then the overall 
hypothesis of equivalence may be accepted^ The four £-rat1os corresponding to 
the four hypothesis were calculated and tested for significance in both 
studies 

Additional correlational analyses were performed in Stu4y II ;:c determine 
the existance of possible differential relationships between the two adninis- 
tration modes of the RAT Form 1 and the four reference measures. Pearson 
product-moment correlation coefficients were calculated between each of the 
reference measures and each mode of administration of the RAT Form 1, and all 
pairs within administrations were tested for significant differences with z- 
trans formations 

Results 

Table 1 presents the descriptive statistics for the two administration 
modes of the RAT Form 1 wder both studies o The means and standard deviations 
for all administrations compare favorably with those reported by Mednick and 
Mednick (1967) in the normative data for college undergraduates. Hoyt internal 
consistency reliability estimates, calculated from the analysis of variance 
format used for the Medley procedure, were found to be slightly lower than odd- 
even split^half coefficients reported by Mednick and Mednick. 

One noteworthy comparison that can be made from Table 1 is the high degree 
of similarity between the performance of the technical education Ss in Study I 
and the baccalaureate program Ss in Stu((y II. Students enrolled in two-year 
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Table 1 

RAT Form 1 Statistics for studies I and II 



Study I study II 

2-year 4-year 

Tech. Education Baccalaureate 

Students Students 





CAI 


Conv. 


CAI 


Conv. 


N 


14.53 


15.39 


15.28 


16.50 


SO 


4.58 


5.33 


5.65 


5.10 


N 


19 


18 


53 


54 


Hoyt 
Reli- 
ability 


.74 


.81 


.84 


.78 



technical education programs are generally believed to be less intellectually 
capable than four-year students. Although many variables were uncontrolled 
between Studies I and II the results would appear to indicate that in at 
least one dimension of intellectual functioning the two-year students compare 
quite closely with four-year students. 

Results of the Medley equivalence analysis are summarized in Table 2. 
Calculation of the F-ratlos used to test each of the four equivalence criteria 
necessitated dividing the overall analysis of variance into separate analyses 
representating each mode of administration, and an analysis representing the 
combined administrations. The first criterion of interest, equality of means 
for the two administration modes, was tested by comparing the variance between 
administrations with the variance for Ss within administrations. The resulting 
F-ratios for both studies were nonsignificant, indicating failure to detect 
differences with respect to the mean performances under the two modes of ad- 
ministration. The second criterion, equality of variances, was tested by 
forming an F-ratio from the mean square for subjects in each of the separate 
mode analyses. Again, the F-ratios for both studies were nonsignificant, and 
the conclusion is that the variances for the two modes of administration are 
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not reliably different The third cf tencn, equality of measurement error 
variances, was tested by an ^-rat^o fonned f^om the error terms in each of the 
separate mode analyses. Here a significant £ was obtained for Study II, but 
since the value of the F was only 1,C6 with 1537 and 1508 df the differe-^rp is 
probably not of great consequencec Thus, the criterions of equal o 
measurement variances also appears to be reasonably satisfied. The final 
criterion, homogeniety of function, was tested with an £-ratio formed by com- 
paring the error between administrations with the error within administrations. 
A significant £ was obtained for Study II indicating that the item difficulty 
rankings yiere not the same for both modes of administration. Items proving 
very difficult under computer administration were apparently slightly easier 
under conventional administration and vice versa. The resulting conclusion Is 
that In Study II the RAT Form 1 was not measuring identical functions under the 
two modes of adminlstratione 

Table 3 presents the results of the addUional correlational analyses per- 
formed In Study II. The correlations between each of the administrations of 
the RAT Form 1 and each of the refere-i-^e measures correspond (In the case of 
the first three reference measures) roughly with those reported by Mednick and 
Mednick (1067). The correlations between the RAT Form 1 and the RAT Form 2 are 
slightly lower than reported in the Test Manual. The fourth reference measure. 
Flexibility, does not appear to correlate with the RATc A'l four correlation 
pairs between ad: I n1 strati on modes were tested for significance, and none of 
the differences proved significantc Thus, there is no evidence to indicate 
that the parallel forms reliability or the validity of the RAT are any dif- 
ferent under computer administration than under conventional administration. 

Discussion 

Medley (1957) indicates that •ejection of any of the four hypothesis 
forming the criteria of equivalent tests constitutes sufficient justification 
for rejection of the overall hypothesis of equivalenceo Strict adherence to 
that policy In the present investigation would lead to the conclusion that, at 
least for verbal measures of creativity like the RAT Form 1, computer-assisted 
test administration is not the same as conventional group administration. The 
following considerations, however, are relevant: 
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Table 3 

Correl.5s.iOni of RAT Form 1 and Reference Measures (Study II) 





• 


Reference Measures 




Mode of 
Administration 
RAT Form 1 


RAT 
Form 2 


Anagram 
Frequency 


Anagram 
Originality 


Flexibility 
Inventory 


Conventional 


.55 


.49 


.38 


.11 


CAI 


.66 


.41 


.34 


-.13 



Note. "All differences were tested for significance using the z-transfor- 
mation. None attained s gnificance at the .05 level. Although Lhe individual 
correlations are not marked for significance, a correlation of .269 (df = 52) 
is required for significance under the conventional administration anT"a cor- 
relation of .286 (df = 46) Is required under the CAI administration. Correla- 
^^on*^ equal to or exceeding these values constitute significant departures from 
.^.^0 at the .05 level. 

1) Statistical significance does not, of course, mean practical signifi- 
cance. Although none of the hypotheses was rejected in Study I, the test for 
homogenlety of function and equality of measurement errors did Indicate 
statistically significant differences between modes of administration in Study 
II. The differences appear to be slight, however, and probably of little 
practical significance. 

2) The correlational analyses performed in Study II Indicate once again 
that differences between iiodes of administration. If present, are Indeed slight. 

3) Since homogenlety of function was not significant In Study I where 
technical education students were used as Ss the effect, even if present among 
four-year baccalaureate degree students, cannot be generalized across different 
ability levels or to different classes of students. 

4) Finally, It is Important to consider that creativity was specifically 
selected as the measurement variable In this Investigation because of Its 
known sensitivity to variations In the testing environment. Thus, even the 
most conservative interpretation of the present findings would not warrant 
generalization to the testing of other mental abilities. 
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In summary, the study reported here does not present any evidence to in- 
dicate that computer-assisted test administration introduces nev/ sources of 
variance that markedly modify the statistical properties of a test as deter- 
mined through conventional administration. 
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PROMPTING AND CONFIRMATION AS INSTRUCTIONAL 
STRATEGIES WITH COMPUTER ASSISTED INSTRUCTION 
EXPERIMENT 1 

Keith A. Hall and Karl G. Bonnan 

Cook and Spitzer (1960) and Stolurow and Lippert (1962) among other 
Investigators have found strong differences between confirmation and prompting 
procedures in learning. The data indicate that prompting enables the student 
to achieve criterion level much more rapidly than does a confirmation mode. 
However, confirmation appears to provide better retention than does prompting 
at differing levels of overl earning, Stolurow and Lippert (1962). It is 
important to note, however, that neither of these studies were done in an 
environment which could be broadly applied in education--the Cook and Spitzer 
study was u very closely controlled laboratory study and the Stolurow and 
Lippert study was done using flash cards with mentally retarded children. 
Computer-assisted instruction seems to provide a measure of control somewhere 
between a hand-administered treatment and a closely controlled laboratory- 
administered treatment. 0ns important consideration is that the computer- 
assisted instruction procedures can be replicated and can be utilized by many 
students in a typical educational setting. 

The effects of treaanent (prompting or confirmation); criterion level 
(1, 3, or 6 consecutive correct responses to each stimulus item); intervals 
between administrations of the retention test (1, 7, or 21 aays); and order of 
treatment (prompting treatment followed by confirmation treatment [P ^ C] or 
confirmation treatment followed by prompting treatment [C -»• P]) on retention 
test scores; instruction time; and number of responses during instruction were 
examined in this study. 



Method 

Materials 

Sixteen geometric patterns were selected for use as stimulus items from 
the American Standard Association drawings for piping layouts on architectural 
drawings. As can be seen in Figure 1, there were rather marked similarities 
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Fig. K Stimulus and response item matrix. 



13 



not only between items but also between the Items and the referents. The 
stimulus items were presented under computer control to the subject on a small 
rear-projection screen and the subject responded by typing the name of the 
item on the typewriter keyboard at a computer terminal. The responses consis- 
ted of two elements: 1) the word or phrase which identified the type of 
fastening and 2) the name of the piping element, e.g.. flanged cross, bell and 
spigot globe valve, and welded safety valve. The prompts and confirmations 
consisted of the entire response components (1 and 2 above) and were typed at 
the typewriter terminal by the stored computer program. 

Procedures 

Subjects in Group P C learned the first eight items in a prompting mode 
followed by a second set of eight items in a confirmation mode. Subjects in 
Group C P received the first eight items in a confirmation mode followed by 
a second set of eight items in a prompting mode. The subjects were randomly 
assigned to Group P C or C P and the items were randomly selected for pre- 
sentation by either prompting or confirmation. Additionally, the items within 
each of the two lists for each subject were presented in a random order for 
each subject. 

The confirmation mode consisted of presenting the stimulus, followed by 
the subject's response, which was in turn followed by confirmation consisting 
of the correct response for the subject to view. This sequence was used for 
each Item in the confirmation mode. In the prompting mode the stimulus was 
presented followed by the prompt (consisting of the correct response) for the 
subject to view who then responded by typing that response on the typewriter 
terminal. These sequences are shown 1n Figure 2. 

On the first three cycles through the eight items in each treatment, the 
strict prompting or confirmation mode was followed for each subject. At the 
end of the third cycle a test cycle was employed which presented the stimulus 
Items one at a time to the subject and waited for the subject to respond. In 
this test sequence, no prompt or confirmation was given to the subject. Each 
item which was answered correctly in the test cycle remained in a test cycle 
until the subject reached criterion level (1. 3. or 6 consecutive correct 
responses) on the item or until he responded incorrectly to the item. When he 
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Fig. 2. Sequence of events for each item presented fn prompting 
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reached criterion, that Item was dropped from the sequence. However, if he 
responded incorrectly to an item before reaching criterion, that item was 
returned to the original instructional mode of either prompting or confirma- 
tion. Each subject continued through the materials until he had reached his 
assigned criterion of either 1, 3, or 6 consecutive correct responses in a 
testing mode for each item. It is important to note that after the first test 
cycle, some items remained in a test mode and some reverted to the original 
instructional mode of either prompting or confirmation resulting in a mixed 
presentation. 

Retention tests were administered one, seven, and twenty-one days fol- 
lowing the experimental treatment. Th; retention test consisted of presenting 
each of the sixteen stimulus items to the subject one at a time and allowing 
him to respond on the typewriter terminal = No feedback was given at any point 
in the retention test regarding the success or failure of the subject on the 
items. 

Subjects 

Thirty-nine college sophomores and juniors in the College of Education at 
The Pennsylvania State University with no background in engineering were 
randomly assigned to the experimental conditions Each subject was paid a 
fixed fee for participating in the experiment. 

Design 

A 2 X 3 X 3 design was employed. Each subject was randomly assigned to 
a treatment order (P C or C + P) and to a criterion level (1, 3, or 6) and 
produced data on each of the dependent variables (retention tests at 1, 7, and 
21 day intervals, instructional time, and number of responses during instruc- 
tion. 

It was expected that prompting would produce faster learning (less time 
and fewer responses during instruction) but poorer retention on the materials 
and that confirmation would produce slower learning (more time and more 
responses duriiig instruction) of the materials but better retention. It was 
expected that criterion level (or the amount of over- learning) would interact 
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significantly with the instructional treatment and retention interval. Fur- 
ther, it was expected that there would be no proactive or retroactive facilita- 
tion or inhibition between the nx)des or sequence of instruction. 

Findings 

Analyses of variance were perfonned to determine the effects of the 
independent variables upon the dependent variables. Table 1 summarizes the 
analyses of the dependent variables associated with acquisition of the 
learning task with order considered. 

The Effect of Order 

Number of responses . The analysis of the number of responses during 
prompting and confirmation yielded no significant differences. However, the 
requirement of homogeneity of variance was not met for the number of responses 
during confirmation. The effect of violating the assumption of homogeneity of 
variance is unknown and therefore precludes any interpretation of the data. 

Instructional time . The an»ilysis of the amount of instructional time 
during the prompting mode yielded an F-ratio of 4.217 which is significant at 
the .05 level of confidence (df » 1,33). The mean instructional time during 
prompting was 37.04 minutes (n « 19) when the prompting items were presented 
first and 29.44 minutes (n « 20) when the prompting items were presented 
second. A similar result was obtained from the analysis of instructional time 
during confirmation although the requirement of homogeneity of variance was 
not met which again precluded interpretation of the data. The effect that 
order hac on instructional time during prompting and the uncertainty of the 
data related to the number of responses and the instructional time during 
confirmation cast doubt on the advisability of pooling all confirmation data 
and all prompting data and ignoring the effect of order as had orginally been 
planned. Therefore, the remainder of the analyses were conducted with order 
of instructional mode considered and kept separate. 
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The Effect of Criterion Level 

Number of responses . Criterion level (1, 3, or 6 consecutive correct 
responses) produced an unexpected progression of number of responses for Items 
learned by prompting (1: X « 26.36, n = 14; 3: I « 45.10, n = 14; 6: X « 34, 
n « 11) the differences all being significant at the .01 level (F-ratIo « 5.564, 
df « 2,33). The violation of the assumption of homogeneity of variance for 
the confirmation data precludes the interpretation of these data. 

Instruction time . Criterion level (1, 3, or 6 consecutive correct 
responses) had the anticipated effect of producing a steady Increase In 
Instructional time for Items learned by prompting (1: Y= 20.48 minutes, 
n = 14; 3: X = 38.11 minutes, n = 14; 6: X = 42.93 minutes, n 11) and for 
Items learned by confirmation (1:7= 22.86 minutes, n « 14; 'i: J = 37.43 
minutes, n = 14; 6: Y = 45.51 minutes, n = 11). The analysis of the effect of 
criterion level on the amount of time during prompting mode produced an F-ratIo 
of 13.675 (df « 2,33) which is significant at the .01 level. However, again 
the violation of the assumption of homogeneity of variance for Instructional 
time for Items learned by confirmation suggests some degree of uncertainty In 
these data. 

Interactions 

Prompting . Table 2 summarizes the analysis of variance for repeated 
measures of retention scores for Items learned by prompting with order con- 
sidered. Criterion level (1, 3, or 6 consecutive correct responses) had the 
anticipated effect on retention with Items learned by prompting, I.e., the 
higher the criterion level the higher the retention scores (1: 7 « 4.45, 
n « 42; 3: Y » 4.57, n = 42; 6: Y » 6c27, n = 33). The differences between 
1 and 6 and 3 and 6 consecutive correct responses were significant at the .05 
level. Although an analysis of retention scores for Items learned by prompting 
across retention Intervals yields an F-ratIo of 3.789 which Is significant at 
the .05 level (df « 2,66) « It cannot be unambiguously evaluated because of a 
significant Interaction between retention Interval and order shown graphically 
in Figure 3. 
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Table 2 

Analysis of Variance 
for Repeated Measures of Retention Scores 
for Items Learned by Prompting with Order Considered 





Sources of Variance 


DF 


MS 


F-ratIo 


Between Subjects 








Order 


1 


13.03 


1.447 


Criterion 


2 


36.88 


4.096* 


Order x Criterion 


2 


5.21 


<1.00 


Error 


33 


9.00 




Within Subjects 








Retention Interval 


2 


6.01 


3.789* 


Order x Retention 


2 


5.15 


3.249* 


Criterion x Retention 


4 


1.66 


1.045 


Order x Criterion x 
Retention 


4 


.26 


<1.00 


Error 


66 


1.59 





significant at the .05 level 



Confirmation . The analysis of variance for repeated measures of 
retention scores for Items learned by confirmation with order considered Is 
summarized In Table 3. The analysis of scores i*.ross retention Intervals 
during the confirmation mode yielded an F-ratIo of 10.138 which Is significant 
at the .01 level cf confidence (df « 2,66). This cannot be evaluated properly 
because of the Interaction with order shown graphically In Figure 4. 
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Fig. 3. Interaction between retention Interval and order for Items 
learned by prompting. 
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Table 3 

Analysis of Variance 
for Repeated Measures of Retention Scores 
for Items Learned by Confirmation with Order Considered 





Sources of Variance 


DF 


MS 


F-ratIo 


Between Subjects 








Order 


1 


3.83 


<1.00 


Cri terlon 


2 


27.95 


2.844 


Order x Criterion 


2 


9.08 


<1.00 


Error 


33 


9.83 




Within Subjects 








Retention Interval 


2 


14.03 


10.138** 


Order x Retention 


2 


6.18 


4.471* 


Criterion x Retention 


4 


.62 


<1.00 


Order x Retention x 
Cri terlon 


4 


.11 


<1.00 


Error 


66 


1.38 





significant at the .05 level 
*S1gn1f leant at the .01 level 



Conclusions 

The data Indicate that prompting Is the better procedure to use for the 
Initial learning. S's who learned by prompting first required less time to 
reach criterion dnd also maintained better retention as Is shown graphically 
In Figure 5. 

The data produced by the confirmation treatment Indicate that some prior 
experience with the materials to be learned was helpful. Confirmation did not 
provide the strong results that were expected on the retention tests although 
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Fig. 5. Retention mean scores for Items learned during the "first" 
treatment. 
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there Is perhaps some contal ml nation In the data resulting from Items which 
were not completely independent of each other making these data difficult to 
interpret. This conclusion Is supported by both Figures 5 and 6 which Indicate 
similar trends for the "first treatment" regardless of whether It was prompting 
or confirmation and similar trends for the "second treatment" regardless of 
whether It was prompting or confirmation. 

Criterion level ha J the anticipated result of Increasing acquisition time 
although again the data are not clear enough to Interpret properly. 

It seems highly desirable to further investigate the findings of Cook 
and Spltzer and Stolurow and Lippert In an attempt to apply them to computer- 
assisted Instruction. A further refinement of the experimental materials is 
necessary along with improvements In the experimental design to provide the 
necessary control In the environment. Computer-assisted instruction offers 
the opportunity to manipulate variables of the kind investigated here which 
should lead to the preparation of Improved instructional sequences. 
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ANALYSIS OF STUDENT PERFORMANCE RECORDS 
Frederick N. Chase and Terry A. Bahn 

Although extensive Information about a student's progress through a course 
can be recorded by the IBM 1500 Instructional system, the need to select, 
organize » and analyze these data for researchers and curriculum development 
specialists has been a need not fully met by the standard 1500 system. The 
single capability provided as part of the 1500 system for the output of per- 
formance data is that of listing on paper a student's "performance records" In 
chronological order. (Appendix A). By this means all data about a given student 
can be retrieved, but In a format 111-sulted for many types of subsequent analy* 
sis. In particular. If one wishes to analyze several students' responses to a 
given question, one must shuffle through each student's listing. If one wishes 
to do further machine processing of the performance data, the re^Sevant Informa- 
tion must again be made machine-readable (e.g«, key punched) by humans, with 
ensuing loss of time and accuracy. 

Recognizing this Inadequacy, many users of 1500 systems have developed 
their own methods of selecting, organizing, and analyzing student performance 
data. All such methods of "data management" known to us as of fall 1968^ were 
Investigated and rejected for one or more of the following reasons: 1) did 
not meet our objectives; 2) was hot compatible with our 1130 based system; 3) 
1132 printer which was used Is too slow to provide acceptable turn-around time; 
4) available documentation was Inadequate; or 5) date of completion of data 
management package was uncertain. 

In view of these problems, the staff at Penn State decided to write the 
performance record analysis programs described In this paper. We have capital- 
ized on our past experience with data management programs (1) and on the avail- 
ability of large computers on the University campus. Very briefly, the programs 
allow one to select, sort, and print data from raw tape performance files. An 
optional statistical analysis Is possible after sorting and before printing. 



'These were Florida State University (2); State University of New York 
at Stony Brook (5); and Stanford University. 



28 



The following sections describe the raw data which may be collected by the 
1500 system for input to the analysis programs, the types of output available 
from the programs, and finally the means by which the output Is obtained. 

Input to the Performance Record 
Analysis Programs 

After a student has responded to an instructional question and the response 

has been accepted and analyzed by the 1500 system, a "perfonnance record" may 
2 

be written on maonetlc tape. Each performance record can contain the course 
name* student name. Identifying name of the Coursewrlter II e£ Instruction 
which accepted the response, student response, date, time, and other Informa- 
tion (3^. The performance record is a permanent report of the status of the 
student at the time of his response. Any user (student or author) for whom 
performance recording was specified will cause records to be written as the 
computer accepts and analyzes his responses. These perfonnance records are 
written one after another, in strict chronological order, on the performance 
magnetic tape. 

Output From the Performance Record 
Analysis Programs 

The Performance Record Analysis Programs provide three categories of out- 
put data for further analysis of the perfonnance records logged during an 
instructional session. The remainder of this section describes these three 
categories at a medium level of detail. 

Category R - Research . In preparation for meaningful output, records for 
one or more CAI sessions are combined. All records for given course segments 
and given dates are then selected from this agglomeration of records. The 



Whether a record Is written depends on whether perfonnance recording was 
requested at the time the student was registered and on the choice made by the 
course author to suppress or not suppress recording of a given response. 

The Input medium for the perfonnance record analysis programs Is tape, 
although the 1500 system Is capable of writing the performance file on a disk. 
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tape resulting from this process can be given to a researcher for further* 
specialized analysis* Such a tape is one (currently the only) output option 
under Category R,(R is intended to suggest research). 

This output could be used as input to certain library programs of other 
computers. 

The second and third categories of output require a sort following the 
selection discussed above. 

Category C - Course Development * In Category C, (C Is Intended to suggest 
course oriented analysis), a sort (by "ep identifier") collects performance 
records for a given part of the course (specifically, for a given answer 
request. I.e., a given e£ Coursewrlter Instruction) . Category C output Is use- 
ful whenever one's perspective Is course-oriented. I.e., whenever one wishes to 
focus on the status of all students as they passed a given point In the course. 

Performance records used for Category C output are selected on the basis 
of a user-specified list of eg^ identifiers ( a1J[ may be specified) and a user- 
specified list of student numbers (alj^ may be specified here also). 

After this selection has been completed, the desired records may be printed 
In full, or compressed Into a condensed form of one printed line for each 
Instructional question, (See Appendix A). 

If the records are to be printed In full, the user may request that any 
combination of the following parts of the performance records be printed: 

1. course name and segment number 

2. student number 

3. ep identifier (the author-specified name of the response request or 
e£ Instruction) 

4. response latency (time for student to respond) 

5. match Identifier (the two character name of the Instruction whose 
operand matched the students' response) 

6. date of recording 

7. time of recording 



Full specifications of the sort are first by segment of the course, then 
by ep Identifier within segments, then by student number within ep*s then by 
date, finally by time. 
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8. response length (number of characters) 

9. number of times this student has previously responded to this eg.. 

10. switches 

11. counters 

12. student response 

Category C output might be used to scan all student responses to questions 
in a course. 

If the one-line summary is requested, each one-line summary contains the 
§2. name, the number of students who responded to that ep, the total number of 
attempts on that question, the mean number of attempts (quotient of previous 
tMo numbers), the mean number of seconds in a response, and the fraction of all 
students responding more than twice to the e^.. This output might be used to 
select for detailed inspection the questions for which the average student 
responds several times before he is allowed to proceed. 

Category S - Student Trace . In Category S, (S is intended to suggest 
s^tudent-oriented analysis), a sort by student number^ collects all responses 
for a given student (Appendix A). Within a given student's responses, entries 
are still ordered chronologically. Category S output is useful whenever one's 
perspective is student-oriented, i.e., whenever one wishes to follow a student 
through the course without regard to other students who may also have taken the 
course. 

Performance records in Category S are selected as in Category C, i.e., by 
e£ identifier and student number. After this selection is completed, the desired 
records are printed in full with a choice as to parts of each record which will 
be printed identical to the choice available under the full print-out option in 
Category C. 

Category S output could be used to debug a course. 



^Full specifications of the sort are: first by segment of the course; 
then by student nunber within segments; then by date; finally by time. 
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Flow of Data During 

Analysis of Performance Records 

At various stages in the process of obtaining student performance records, 
the data is manipulated by four programs. These programs will be described 
briefly In this section. 

After performance records have been collected on a number of small tapes 
(approximately 600 feet), Program 1, an IBM 1130 Disk Monitor FORTRAN program, 
Is used to stack these tapes onto a large tape (approximately 24000 feet). In 
addition, this program changes the variable length CORE IMAGE records Into 
fixed-length (232 byte) records which are in a simple format used by later 
programs (Appendix B). This stacking is necessitated by the fact that 1500 
tape performance files cannot be reopened for additional recording sessions 
subsequent to the first recording session. 

When Program 1 is initiated, it checks the output tape to be certain that 
It has a valid IBM 36Q/0S label. It then checks the input tape for a valid IBM 
1500 OS performance tape label. At the end of the input tape, the operator Is 
given the option to mount another Input tape or to terminate the job. If a 
new tape is mounted. Its label will be checked before stacking continues. The 
program Includes a number of error recovery routines to handle label, record 
and tape errors. 

Program 1 has 3 major subroutines: MAGTP Is a modified version of MAGTA, 
an IBM Type III FORTRAN subroutine for reading magnetic tape (4). MAGTP differs 
from MAGTA as follows: a) EOF and tape errors are Indicated In an additional 
argument so that they may be handled by the main program rather than by the 
routine Itself; b) one rather than two word integers are used; and c) back- 
spacing after an unrecoverable read/write error Is not automatic so that 
damaged records may be skipped^ EXPAN Is a FORTRAN callable subroutine written 
in IBM 1130 Disk Monitor Assembly Language. It Is this routine which converts 
the raw Input records Into formatted, fixed-length output records. KLOZE Is a 
FORTRAN subroutine which writes IBM 360/0S Trailer labels on the output tape 
when stacking Is terminated. 

Program 2 Is FORTRAN IV main program run under IBM OS/360. Its purpose Is 
to select records on the basis of course name, segment number(s), beginning and 
ending date, from the master tapes created by Program 1. These records are 
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written on a tape which will be used as input to Program 3. If the segment 
number, beginning date or ending date parameters are omitted, they will be 
ignored as a basis of selection, i.e. all segments, all dates, or all dates 
before or after a single specified date will be selected. 

Program 3 is the IBM OS/360 Sorting/Merging Program. It can sort the 
records selected by Program 2 in any order on any data field(s). However, two 
major sorts have been used to date: 1) time of recording within date within 
student number within e£ identifier within course sediment— this sort is used 
for Category C output and 2) time of recording within date within student 
number within course segment—used for Category S output. 

Program 4 is a FORTRAN IV main program. It povides a full listing and/or 
statistical summary of the sorted performance records which may be further 
selected on the basis of student number(s) and/or eg. identifier(s) . As in 
previous selections, if either parameter is omitted, all student numbers #nd/ 
or egi identifiers available for the course are included. 

Future Developments 

The latter three of the programs described above are currently operating 
under OS/360 on an IBM 360/67 computer. The source language is Fortran IV. 
The program package should thus be adaptable to any larger computer with a 
Fortran compiler and a library sort program. A possible problem in using a 
computer other than an IBM 360 is the need for Fortran character comparing 
subroutines. 

Plans for future extension and improvement of t;he programs include pro- 
vision of an option in Category R for output which is suitable for direct 
input to library statistical analysis programs. 



Appendix A 



This appendix contains a sample section of Coursewriter II code and the 
various types of output which can be generated for students who encountered 
that code while on-line. The interrelations of the various types of output 
are 11 lustra ted. 

Fig. 1. Listing 

Fig. 2. Performance Output 

Fig. 3. Category S Output 

Fig. 4. Category C Output 

Fig. 5. Summary 
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Fig. 1. 

The course listing (Fig. 1.) shows part of a sample frame from a course 
segment called stat-5. The frame begins with gr op code. Then the computer 
is instructed to enter and process the student's response (eg^ op code). The 
author has specified an e£ identifier of 0520000000. Following the e£ op code 
are the response analysis instructions. Various possible answers are tested 
for. If one of these is found, appropriate action is taken. If the student 
types 74.3, he is given the message at 167-17 and branched to label 180. 

Statement 3 of this program (dt 0,0/2 ...) causes the numeral 20 to appear 
on the cathode ray tube. This refers the student to the following problem in 
his handbook: 

Problem 20: Given a normal distribution with n « 302, how many 
cases will have z scores which deviate from the 
mean z of zero by an absolute value of 1.16 or more? 

The student has been asked to solve the problem prior to his session at 
the terminal and should be ready to input his response. 
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Fig. 2. 



A student, whose number Is u9, encountered the course frame on January 15, 
1969. The IBM-supplied perfor program produced the output shown in Fig, 2. 
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Fig. 3. 



In Fig. 3, Is a page from our Category S output. The listings In Fig. 2 
and Fig. 3 are seen to have the same structure and Information content, the 
primary differences being In format and In the means by which the listings were 
produced. We feel that the format of our Category S output Is considerably 
more legible, due primarily to the columnar arrangement of data and the sup- 
pression of unneeded labeling. The perfor output was done on the 1500 printer, 
while the Category S output was produced on a high-speed printer at Penn State's 
Computation Center. 
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Fig. 4 and Fig. 5 

The Category C (course-oriented) output In Fig. 4 shows Student u9's one 
performance record and also at the same place In the listing all other per- 
formance records for e£ number 0520000000. The one-line summary (Fig. 5) 
gives for e£ 0520000000 a condensation of the data displayed In Fig. 4. 
Reading from the left, we note that for this eg^, six students collectively 
gave eleven responses. About sixteen per cent (1 out of 6) of the students 
tried more than twice. On the average, a student tried 1.8 times on this 
frame. A typical student spent about 50 seconds responding. 
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Fig. 4, Category C Output - Continued on following page. 
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Append'iv B 

Penn State Fixed Length 1500 
Performance Records Format 



Bytes 


Contents 


Number of Bytes 


Format 


1-2 


Number jf 2-Byte 
Words in Student 
Response 


2 


Binary 


3 


Restart Point Passed 
5wi ten 


1 


EBCDIC 


A 
•» 


Restart Performed 
"Switch" 


1 


EBCDIC 


5 


Proctor With Student 
"Switch" 


1 


EBCDIC 




Time of Recording 


4 


Binary 




Student Number 


4 


EBCDIC 


14-11 


Course Name 


5 


EBCDIC 


19-29 


Segment Number 


2 


Binary 


21-22 


Month 


2 


Binary 


23-24 


Day 


2 


Binary 


25-26 


Year 


2 


Binary 


27-36 


EP Identifier 


1? 


EBCDIC 


37-38 


Match Identifier 


2 


EBCDIC 


39-138 


Student Response 




EBCDIC 


139-29l> 


Counters 0 Through 30 


62 


Binary 


2gil-232 


Switchts 9 Through 31 


32 


EBCDIC 
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SURVEY OF COMPUTER SCIENCE LITERATl'RE RELEVANT TO CAI 

Paul Rowe 

Introduction 

The purpose of this paper is to review and discuss some computer science 
articles published in scientific journals over the past ten years. Many of the 
ideas and developments in these articles are quite new; several were published 
in 1967 and 1968. The goal in discussing these articles is an attempt to evalu- 
ate their potential usefulness to CAI. The word 'potential' is used because 
even the best idea is not useful unless it is implemented, and the question of 
adoption of the ideas in these articles will depend on decisions made by many 
different people. 

The author of this paper does not claim a complete literature search in 
this area, although in an attempt to cover this area of computer science, a 
very extensive literature search was made that Involved the Pattee Library, 
the Math Reading Room, the Engineering Library, the Physics-Cnemistry Library, 
all on the Penn State University campus. In an attempt to cover this area of 
computer science, some private correspondence with computer scientists at other 
universities was Initiated; however, very little new material resulted from 
this source o 

Computer Languages 

There Is an extensive amount of research and study currently being conduc- 
ted on both natural and artificial languages o This work ranges from the mathe- 
matical study of context-free languages to empirical studies of natural 
languages. Since communication between man and computers utilizes artificial 
languages, no further mention of natural languages will made in this paper. 
The set of computer languages is a proper subset of artificial languages. The 
reason for this Is that there are many artificial languages whicn have interest- 
ing and Important theoretical properties but have very limited capabilities as 
man-computer communication languages. 

Computer languages generally have been developed by one of two different, 
but not entirely unrelated, methods. The following disucsslon will be an over- 
simplification of the two methods. The first method consists of specifying the 
source statements as to form and purpose, A processer 1s then written to 



48 



process the source statements into machine language statements that will 
accomplish the original purpose of the source statements. In the most elemental 
form of this method, no attempt is made to have the source statements contain 
any theoretical properties. The second method consists of specifying the 
theoretical properties cf the source language as to form and purpose, but they 
must fit the theory. Finally, as In the first method, a processor Is written to 
obtain machine language statements. It Is obvious that one could develop a 
computer language by a combination of the two methods. For example, a subset 
of the source statements could be required to be context free, and the rest of 
the source statements specif ltd without regard to any theory. ALGOL Is an 
example of a computer language developed by the second method. There Is merit 
to both methods. The first method is less restrictive on the specifications 
of source statements. Use of the second method gives greater assurance that 
the language developed will be unambiguous and have other properties that are 
considered desirable by the people developing the language. 

There have been two languages developed specifically for CAI ( Coursewrl ter I 
and Coursewrlter II). Other computer manufacturers have developed their own 
CAI languages for their computers, but these will not be discussed In this 
paper. Typical of CAI languages developed thus far, each one has been developed 
for a particular computer. Since Coursewrlter was developed with a specific 
computer In mind (presumably IBM 1800/1130) any change to a larger, more capable 
computer will probably necessitate at least a modification of the present soft- 
ware If not the adaption of a different language or the development of a new 
language. There Is a need for using more capable and faster hardware to bring 
down the cost of CAI. Even with the present system (IBM 1500), there Is need 
for modification of the present software to enhance the use of CAI. Therefore, 
there Is a need to look at various existing languages which might be capable 
of adapting to CAI and also the possibility of developing a new language for 
CAI. 

There are several existing languages which might be adapted to CAI. One 
that hes already been used with IBM 360 systems Is Iverson's 1962 APL. Text 
handling capabilities were added to APL. Several languages have been developed 
for human-computer communication In an Interactive mode. These languages 
either do or could be modified to use the various CAI Input-output devices. 
One of the most recent of these was developed by Kulsrud (1968). Actually 
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Kulsrud did more than develop a new language for graphic display and other 
Input-output devices; he showed that such languages could be obtained by syntax 
specification and that these languages and their processers could be acquired 
relatively quickly. If the Idea of a new CAI language Is ever considered. It 
would be worthwhile to make a thorough study of this and similar languages. 
Other languages that one might consider are Dialog (Cameron, 1967) and Pose 
(Schleslnger, 1967). There will probably be more new Interactive languages 
developed In the near future. 

If a larger and more capable computer than the IBM 1500 (1130 base) Is used 
for CAI, perhaps It would be best to develop an entirely new language. If the 
present research on the theory of computer languages continues. It should soon 
be possible to get desired properties of the language by specifying the theory 
that the language must satisfy. Several articles report progress that has been 
made In this direction. For example, the article on context-free languages by 
Par1kh» 1966. Thus It Is known that a context-free grammar does not always 
generate an unambiguous language. Ways of modifying the grammar are known so 
that an unambiguous language always results. However, this Is sometimes too 
restrictive on the resulting language. Research In this area still appears to 
be active. Ginsburg, Grelbach, and Harrison have done much work on the theory 
of languages (Ginsburg, 1967; Grelbach, 1968; and Ginsburg, 1968). Other 
articles of Interest Include Aho (1968) and Knuth (1968). To Implement a new 
language. It Is necessary to obtain a processor^ The ease with which this can 
be done Is an Important consideration. Some recent papers have an Important 
bearing on this» The article "Stack Automata and Compiling," by Ginsburg, 
Grelbach, and Harrison (1967), and the small book A Syntax Oriented Translator 
by Ingeman (1966) are both relevant. There are many more articles that will 
not be listed In this paper. Perhaps the most Important contribution to showing 
how processors could be obtained quickly was "A Formalism for Program Translation 
by Sklansky, Finkelstein, and Russell (1968). They describe a formalism for 
representing sequences or networks of p'^^gram translations and compiler transla- 
tions. It should be easy to extend their formalism to other type processors. 
Their formalism permits very complicated translation sequences to be followed 
and extended easily. 
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Theory of Programing 

Not as much work has been done on this subject as on languages; however, 
some papers have been published and interest seems to be increasing. Some of 
this may be more Important to CAI than the theory of languages. Two rather 
general articles on the theory of computation were published by McCarthy 
(1962 and 1967). The article "Assigning Meanings to Programs" was published 
by Floyd (1967). Its purpose war to provide a basis for formal definitions of 
the meanings of programs, such that a rigorous standard is established for 
proofs of correctness, equivalence, and termination of computer programs. The 
results and methods of this article should be applicable to programs written 
In most computer languages including present and future CAI languages. A 
method for systematic error analysis of computer programs was developed by Miller 
(1963). The method uses concepts from graph theory and Boolean matrices. The 
method requires a flow chart of the program, which Is a drawback for applica- 
tions to CAI. This is true especially when on-line authoring Is done. However, 
programs can be written that will analyze a program and produce the equivalent 
of a flow chart. 

A paper by Karp (1960) discusses the application of graph theor' to 
digital computer programing. He defines a graph-theoretic model for .he de- 
scription of flow charts and programs. He shows that the properties of direc- 
ted graphi and of the associated connection matrices can be used to detect 
errors and eliminate redundancies in programs. He also shows that these pro- 
perties can be used In the synthesis of composite programs. He then expands 
the graph-theoretic model to take into account frequencies of execution of 
portions of a program and solves a problem of optimum arrangement of a program 
in storage. This paper could be very important to CAI as the programs written 
in Coursewriter II or any other CAI language become more complex and sophisti- 
cated. However, it will take an effort to master the necessary techniques. 

There are two more articles which are relevant to a discussion on the 
theory of programing. The first is a paper by Marimont (1960). This is a 
rather elementary paper on application of directed graphs and Boolean matrices 
to computer programing. This is probably a good starting place for a person 
who wants to learn the theory of programing. The second article was written 
by Martin and Estrin (1967). They discuss estimations of computing times for 
a program on a given processing system and the probabilities of reaching 
vertices in a graph model of compttations. 
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Segmentation of Programs 

It is difficult to say at this time whether the papers published on this 
subject will even be of practical importance to CAI. Programs written in 
Coursewrlter II are read from disk into meniory a segment at a time; therefore, 
segmentation of programs Is used by CAI. However, one would have to try the 
ideas published on program segmentation to determine if any increase in ef- 
ficiency were obtainedc Schurmann (1964) wrote a paper on the use of graphs 
for the analysis of the distribution of loops In a program. Segmentation of a 
program Is most efficient when the least number of loops are cut in the segmen- 
tation process. The problem of analyzing graphs by connectivity considerations 
Is discussed In a paper by Ramanoorthy (1966) This is quite an advanced paper 
and requires a good mathematics background for complete understanding. An 
article by Marimont (1959) contains some ideas relevant to this topic. The 
problem she discussed concerns the checking of the consistency of precedence 
matrices. A short, but very good paper, on segmentation of programs was written 
by Berztiss (1968). 

Paralle l Computation and Parallel Computers 

This is a controversial topic at the present time^ The central idea Is 
tha modular or parallel computers could be built to share a large memory. A 
supervisory unit would assign tasks to the modular computers, but each computer 
would contain its own arithmetic and logical unite This would be very good for 
CAI applications since several terminals could be serviced simultaneously. 
There are several articles on this subject and some are quite long. Much of 
the mathematical the'^ry of parallel computation was developed by Dorn, Hsu, 
and Rivlln (1962)o Their paper was intended mainly for numerical analysis, but 
the material on rearranging programs for efficient parallel computation Is 
generally applicable to any computations that can be done c1>^uUaneously . A 
general mathematical model for modular computers is developed by Wagner (1964). 
His Is a very long paper, and the only importance to CAI Is to keep In mind 
that people are working on parallel computers and in the future such computers 
could be Important In applications of computer science. Karp and Miller (1966) 
apply graph theory to the description and analysis of parallel computations. 
They obtain some very Interesting results, both theoretical and practical. A 
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very good article by Schwartz (1963) gives a procedure for parallel sequencinu 
with a choice of machines. In general this paper Is easy to understand and 
has many good Ideas. 

One of the main problems In parallel programing is to determine all the 
computations and especially the least number of computations that must be 
performed before a given computation can be initiated. A paper by Hu (1968) 
discusses a decomposition algorithm for finding the shortest paths In a net- 
work. The author states that the algorithm can save on both amount of computa- 
tion as well as storage requirements. An article by Schwartz (1961) describe! 
an automatic assignment procedure for use for programing parallel computers. 
This paper Is quite easy to read and should be a good starting place for one 
wanting to understand parallel programing and parallel computat'Jons . 

Much too short a time was spent on ^ach topic to be In any sense complete, 
and the many Interconnections between the topics were barely considered. One 
topic was entirely omitted. Thus, none of the articles on man-machine Inter- 
actions In a learning mode were Included. There are articles due to come out 
soon on most of the subjects mentioned in this paper. Therefore, a revision of 
this paper should be made In the near future. This revision should discuss 
the published articles more thoroughly and Include new papers on any topic 
relevant to CAI. 
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■ 

I PROGRAMED PATTERNS: 

■ A COMPUTER-ASSISTED APPROACH TO SPELLING 

H j Helen L, K. Farr 

B James J. Kelly David D. Palmer 

HB. 

H About the time that computer-assisted instruction (CAI) was first being 

I developed, social and technological circumstances in the nation prompted 

B ' Congress to increase federal appropriations for vocational and technical educa- 

H tlon. The almost simultaneous occurrence of these two events led to the 

H I development of the spelling course described here. One of the major problems 

vocational educators faced was that of interesting and motivating students 
H who, in many cases, had experienced years of academic frustration and frequent 

H failure. This problem was especially acute in the subject matter called 

H "English" (or "language arts" or "communication skills"). Within English, both 

I ' academic and vocational teachers reported a general deficiency in spelling 

H - skills among their post-high school students. So, because there was declared 

H ^ need for spelling Instruction and because spelling is a comparatively easy 

H portion of English to deal with in CAI, spelling was selected as appropriate 

H I subject matter for concentration. 

H An initial series of short courses (Hoqan and Farr, 1966) had successfully 

H I d^nstrated (Farr and Hoqan, 1967) that CAI could be used to teach spelling 

H to post-high school vocational*technical students. Those courses were presen- 

H I ted on an IBM 1410 system, using typewriter terminals equipped with slide 

H projectors and tape recorders. Therefore, when an IBM 1500 system using stu- 

H I dent stations equipped with cathode ray tubes (CRT) and image projectors 

H I replaced the 1410 typewriter terminals, the spelling course had to be trans- 

H . lated for use on the new system. However, since the 1510 terminals presented 

H { much richer Instructional opportunities, it was decided that the spelling 

H course should be redesigned and restructured. 

I I The primary purposes of this revision were to maximize the known capa- 

H bill ties and characteristics of CAI and to explore untried potentials of the 

H I 1510 terminals In teaching a thoroughly familiar—if unmastered— skill subject 

H to young adults who had a history of failure or underachlevement In that 

H 1 subjecto For example, we considered the often criticized "Impersonal" charac- 

H ' terlstlcs of the computer, and we decided that In teaching adults a subject 

^ I 

ERLC' 



56 



"they should have learned years ago," the absence of a human teacher-grader 
might be a distinct advantage. We also considered the computer's well-known 
capacities for presenting information according to a strategy, for accommodating 
student options, and for quickly analyzing and storing various amounts and 
kinds of data. These capacities could be extensively exploited in our spelling 
course when used in conjunction with teaching practices such as organization, 
discovery, prompting, and reinforcement. But most of all, we considered the 
distinctive visual capacities and characteristics of the CRT (essentially a 
television screen) for presenting a subject matter that most of our students 
could be expected, on tht oasis of past experience, to regard as stale, dull, 
and perhaps, as hopeless. 

We recognized that negative student attitudes like these were fundamentally 
as challenging to course authors as the 1500 CAI system itself was. Therefore, 
a secondary purpose of our revision was to try to dispel the unfavorable 
attitudes associated with a spelling failures by making our presentation of 
the subject as different from past presentations as possible: technically, 
through CAI; and substantively, through restructuring the subject matter itself. 
To this end, we examined the content and structure of more than 20 spelling 
texts for elementary and secondary students. As a result of examining those 
textbooks, we decided to structure our revised course according to relevant 
linguistic principles, namely, componential analysis, pattern recognition, and 
rule formulation. These fundamental linguistic principles were not only 
appropriate to the five common spelling errors that we selected for instruc- 
tion, but they also seemed particularly suitable for implementation on the 
1510 terminals. 

Concisely then, in our revised spelling course for post-high school 
students, we set out to innovate as much as was educationally and technically 
feasible in teaching an "old" subject to discouraged and/or bored young adults. 

The revised program is nearly finished at this time. In the first half 
of 1969 it will be completed, thoroughly debugged, field tested, and tried 
out jn toto. On the basis of preliminary student performances, some changes 
have been made, and others may be made after the field, testing. The needed 
changes indicated and made so far, however, have been minor ones. Consequently, 
it is unlikely that the basic goals, rationales, strategies, structure, or 
presentation of the course will undergo extensive changes in the foreseeable 
future. 
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Aspects of a CAI^ Spelling Course 

In planning and developing this course, we attempted to consider at all 
times the total learning situation and a^l of its aspects that might be 
relevant to our purposes. That is, before a decision was incorporated into any 
part of the course, we evaluated the usefulness and probable impact of the 
proposal from each of the following aspects: the station environment, the 
visual channel, the student-machine relationship, the educational objectives 
and rationale, and the CAI-CRT potential (1 e., author objectives). Although 
these aspects are described and explained serially here, they were dealt with 
concurrently while we worked on the course 

Environmental Aspects 

In our CAI laboratory, the student does not usually see the IBM 1500 
computer (where the spelling course is stored) unless he visits the machine 
roonip Instead, he communicates with the computer by using student station 
equipment in a carrel-like arrangement* This location, called a "station," 
is roughly comparable to a student's desk or seat in a regular classroom, 
although the station also includes the physical tools of learning, which in a 
classroom might be things like paper, pen, chalkboard, etc. As he sits at his 
station, the student sees the following equipment. 

1) Directly in front of him, on a desk-like surface is an electric 
typewriter keyboard which the student uses to type information to the computer. 

2) Above the typewriter keyboard, a cathode *^ay tube (CRT) is mounted. 
The CRT looks like a television screen; but unlike an ordinary, home tele- 
vision screen, the tube permits two-way electronic communication between the 
student and the computer 

3) Attached to the side of the screen mounting is what appears to be a 
good-sized plastic pencil with an electrical cord attached to its eraser end. 
This object is a light pen and is used by the student to indicate his answers 
to the computer. When he touches the point of the light pen to designated 
areas of the screen, it is stimulated by light beams from that area of the 
screen and registers the impulse in the computer. In the 1500 system the light 
pen serves as a pointer rather than as a writing instrument. 

4) Eventually, sound equipment will be installed to play audio tape 
recordings. Although it is not yet available, it is mentioned here because 
spelling courses often use aural presentations, especially in testing. 
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5) Students on this course are supplied a pencil and scratch pape'* to 
'jse ir they choose, since many older people seem to partly judge correct 
spelling by a kind of "automatic writing," namely, by a kinetic process. 

6) Although it may seem inappropriate to classify a hurnan procto- as part 
0? th(; environmental equipment at a CAI station, that is what she basical'y is 
for th-'s course. Aside from checking the student in, assigning him a station, 
and showing h^m how to sign on initially, she has no special duties to per- 
•■"ooti ^or our adult spelling students. 



V 'sua - Channel Aspects 

As the equipment listed indicates, this course uses the visual mode of 
communication almost exclusively (students type some responses). All of the 
piateHal presented appears only on the CRT, because we decided to capitalize 
on the black-and-white television screen's capacity for movement rathe'- than 
on the image projector's capacity for displaying colorful but static visuals. 
'.o"!or s often used in spelling textbooks and so might be fairly familiar to 
OK o'der students. The occurrence of moving words and statements is much 
no'-e "are. however, and so we chose to explore how movement within the textual 
rnate»"ai could be used in teaching spelling. 

Because the visual channel of the 1510 station was being explored, the 
njtho-s used the on-line input method almost exclusively, so they could 
rimed ate"!y inspect what they had just put into the computer. That -^s, a^ter 
planning and cod^"ng, on paper, a portion of material, the authors themse^es 
used the typewriter to input the information to the computer. Then they had 
-he compute- display that portion of material on the screen; and, where it was 
neces.'^ary, they decided on changes, noted flaws or errors, and replanned the 
p-esentatTon, Macro programs (i.e., computer operations that recur frequently 
'■n a course) were, ar they must be, put into the computer from punch-card decks. 

If the visual motion dimension of CAI is under scrutiny, and if one -s 
attempting to investigate the innovative use of that dimension, on-line input 
s, in the judgment of the authors of this course, the most effective means of 
""nput Card input is certainly faster since it uses system time more effi-- 
inenfy. but for exploratory programing of the type accomplished in th^s course, 
n ^s a second-best method of Inputs Once the potentials of the computer- 
rontroied 1510 screen have been explored, and the desired effects achieved, it 
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is no longer so important for authors to do their own inputting on-line. 
Nevertheless, it seenis to us that the on-line method of input may alv/ays have 
significance in imaginative, innovative CAI programing where the goal Is not 
to transpose the text and format of a book on to a screen. Furthermore, once 
an author has taken the time necessary to investigate, develop, and implement 
a display technique or arrangement, it is a relatively simple and quick process 
for authors of other courses (often in different subject matters) to study, 
revise and adapt that technique for their own purposes. For, if what one 
author devises has application in other subject matter areas, it is obvious 
that a great deal of effort, system time, and money are saved by the users and 
modifiers of what the first author did^ Therefore, the open and sustained 
exchange of ideas, experience, and cnticisms among capable authors, programers, 
graphics personnel, and computer operators is as beneficial to the development 
of the technical aspects of CAI as are some changes in the technology itself-- 
especially during early development. 

If we had not recognized the challenges to our ingenuity and our effective- 
ness in trying to teach an "old" subject to "old" students, we might not have 
felt the necessity to explore the visual capacities of the 1510 terminal so 
thoroughly. Realizing the challenges, however, we were obliged to investigate 
and use techniques that might, according to some critics, seem to be sheer 
gimmickry. Unblushingly, we admit that some of our techniques may indeed border 
on gimmickry^ But if these techniques help us capture the attention of jaded 
students so they learn what they have not learned from a variety of other 
teaching methods, then we must acknowledge that these are also pragmatic tech- 
niques. Behavioral objectives are important elements in CAI; for this course, 
our objectives included basic technical-visual explorations by the authors and 
improved spelling performances by the students. 

Not all of the visual display techniques used in this course are unique 
to CAI; but most of them can be duplicated in teaching, only by using motion 
pictures, which, of course, have no provision for interacting with the student . 
The distinctive display techniques we incorporated to various degrees and for 
various purposes include the following: flashing letters and letter groups; 
moving letters and letter groups; horizontal, vertical, diagonal, and scattered 
letter and word arrangements; gradually appearing and disappearing text, a word 
or a line at a time; significant spacing and placing of text on the screen; 
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underlines, box frames, filled boxes, and other special visual cues; type in 
di^'ferent sizes; special phonetic dictionary symbols; and pauses of varying 
^lengths , 

Concisely, in this course, the static page of a spelling text, that our 
students were accustomed to studying from, was replaced by a dynamic screen. 

Student-Machine Aspects 

The CAI system seemed to provide an unusually promising situation for 
teaching spelling to post-high school students for the following reasons: 

1) For vocational and technical students, almost any activity involving 
a computer seems to have a fascination. For other older students the notion 
of being "taught by a computer" seems to produce curiosity and a somewhat awed 
amusemento In both student groups, therefore, the very concept of CAI provides 
a set for experiencing something new, unexpected, and unpredictable. The 
advantage of inducing such attitudinal sets in adult students who are about 

to once again encounter an "old" subject should not be underestimated. 
Further, this advantage should not be devalued as a one-time, novelty effect. 
Quite the contrary, if students experience— perhaps for the first time- 
success on this course, we can presume that they would have a favorable attitude 
toward it, and therefore, perhaps toward other CAI courses. 

2) Everything that links the student with the computer is called the 
"interface"; (i.e., his terminal or station with its equipment, and the course 
he takes), in most cases, the interface is quite unlike the equipment and 
materials that older students are used to in studying spelling. So as the 
student approaches it initially, the interface does not evoke memories of past 
failures. If he is at all able to conceive of a fresh start on his spelling 
problems, the 1500 student station and its use in this course hold promise for 
that fresh start. 

3) The on-line experience of CAI provides a particularly beneficial 
experience for older students who are used to "making mistakes." The apparent 
absence of personal, human criticism sharply reduces the likelihood of embar- 
rassment and shame that these students have usually come to expect in class- 
rooms. The absence of these presumably destructive reactions should promote 
learning— especially when the subject matter is admittedly elementary for 
adults, 
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4) The individualized nature the CAI prog»^am and equipment encourages 
a student to stick with a problem or an t nstructional segment until he has 
mastered it. The shame, felt m a classroom, wnen a student does not know an 
answer is minimized in CAI, because his classmates do not know about his 
ignorance or mistakes. The embarrassment he may have felt when he "held up 
the whole class" with his errors or ignorance simply does not occur in the 
individualized setting of the CAI terminal 

5) The mechanical character of CAI provides a very satisfactory scape- 
goat for an adult student, frustrated by repeated efforts to learn a given 
unit of relatively low-level information Without generating any guilt 
feelings, he can freely criticize the "dumb machine," argue with it, disapprove 
of its teaching, or the phrasing of its questions, and if he is really wrought 
up, he can walk off in a huffc AH these things he can do without censure or 
retribution in CAI— but not in the regular classroom^ 

6) The tendency of students to anthropomorphize the computer probably 
results from the two-way communication that can take place in CAK Thus, even 
if an unusually irritated student were to sto»^m away from his terminal it is 
far more likely that he would return to try again than it would be if he had 
been using a conventional textbook or wo'-kbook based upon discrete, one-way 
communication In other words, the pertinent CAI feedback tends to produce a 
sense of closure a desire for closure, depending on its content. Except for 
a few programed ones, no spelling text ever told him his answer was right or 
wrong. And with programed texts, he was still left to check his own answers by 
using his admittedly shaky powers of spelling discrimination^ The intrinsic 
interaction between this CAI program and the student compels him to be more 
deeply involved than he probably would be with a spelling text. Besides, who is 
going to admit that he was bested by a machine? For no matter how man-like CAI 
may sometimes seem to be, the student knows that it is still a machine. In 
short, the ambivalence that a CAI experience can generate may be advantageous. 

7) Some of us have wondered how remedial students might react to being 
assigned CAI after they had failed to master material in class. We wondered 
whether they might feel that the teacher had given up on them and abandoned 
them to a machine--one that successful students might soon label "the dummy 
machine." But so far, we have no indications that our post-hiqh school students 
(who must be classified as remedial students in spelling) have had these nega- 
tive feelings about being "turned over to CAI." This fact may be due to some 
of the psychological factors mentioned above (e.g., not having to struggle un-- 
successfully in front of other people) and/or to the student's realizing (as 
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I"- wa'ks and out) that whereas he may be taking a remedial course, students 
<• : other stations in our laboratory are not. Hence, he has no grounds fo- 
f'>sjm''ng that the machine stigmatizes him. We want to stress that while we 
d-e concerned with the advantages of using CAI to teach what may be considered 
n rnod^al material to older students, it should be clearly understood that 
^ .wdMtion is by no means the only (or the chief) domain of CAI. Our coj-se 
wci designed to help solve special learning problems for a particular student 
.lojp There are many other CAI uses and advantages that could be cited, but 
tie/ ^"6 not relevant to the discussion of this course. 

in sunmary, for our students and our course material, CAI seems to offer 
r. i-Thirous advantages over other teaching methods. Because some of the student- 
rncMine aspects are intangible or largely unmeasured does not decrease the^y 
P'jss^b'e bearing on learning. Aside from a limited sampling of self-reported 
j ;udent attitudes, there is little empirical evidence about the student-machine 
(i mension of tha CAI situation. Nevertheless, that little evidence, combined 

V tt- ubse->/at1ons and logical deductions, indicates that (contrary to the fears 
c ^ some, CAI is not dehumanizing; and it need not be cold, impersonal, or in- 
i.^f-^erably rg^d= Realistically, for those who are in a plight similar to chat 

V our spelling students (i.e., not being taught a required skill in class 
tt'Ciuse it is regarded as too elementary for that level )-for students in a 
I --oht like this, an effective CAI course may provide a very gratifying ,3nd 
5<>t,sfy^ng experience. In such circumstances, machines may be more hinane than 
S'*me humans. 

£ ^iuca t'nna ' Aspects 

Irc idnional Approaches . Spelling has usually been classified as a sk^^l 
f.jbject to be acquired in the elementary school. Generally, it has oeen taught 
ill three ways (or in combinations of these): from teacher-assigned lists or 
wurds, from rules assumed to cover major spelling problems and/or their ex- 
ceptions, and from student-compiled word lists based on listening, reading and 
mrled errors on written work. Progress has usually been measured according 
to a test-teach-retest pattern. Test items are presented in either visual or 
aura^ form, rarely in both foms together. Students are required to write 
even words correctly, and/or to select correctly or incorrectly spelled words 
trori a given group. Prerequisite skills for spelling (with varying criterion 
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levels depending on the grade level) have usually been considered to be the 
ability to read, write, memorize, and discriminate visually, aurally, and/or 
kines thetically. 

Usually, when students had spelling difficulties they were advised to 
memorize rules and/or lists of woi^ds dealing with their difficulties. Frequent- 
ly such advice ended with the admonition that since there were so many excep- 
tions to the rules, the students would just have to memorize the spelling of as 
many words as possible. Teachers said they were sorry, but that was how English 
spelling was: irregular, erratic, senseless Consequently, poor spelling 
students tended to view the task of improving their spelling skills as an in- 
trinsically hopeless one For, if they memo^^ized 50 words containing ie or ei 
combinations but were tested with words which weren't on their 50-word list or 
happened to be exceptions to the "i^ before e" rule, tbey were as likely to fail 
the test as to pass it. 

On written assignments, English teachers commonly marked spelling errors 
and often tallied them, deducting credit for the number and/or kinds of errors, 
Thus» on a 250 word composition. If a student ^n the upper school grades mis- 
spelled eight words, the composition might be given a failing grade, regardless 
of its content or style. Or, if two grades were given (one for content, one 
for skills) teachers often weighted the grades equally. In almost no case did 
anyone take note that on a 250 v/ord composition a student making eight spelling 
errors had spelled 242 words correctly. In other words, correct spellings were, 
ordinarily, neither noticed nor rewarded with bonus points. 

Spelling errors nearly always seemed to produce negative teacher criticism 
and subsequently negative student affect toward spelling. Apparently, at every 
level of schooling beyond the primary years, correct spelling has been, in fact, 
implicitly regarded as part of each student's entering skill behaviors, regard- 
less of the explicitly listed objectives and criterion levels for different 
grades. Furthermore, 1n general conversation, when a speaker wants to indicate 
that someone epitomizes ignorance or stupidity, he is quite likely to say, 
"She's so dumb, she can't spell 'cat'." Thus, outside the classroom the lack 
of spelling skill has traditionally been associated with general inadequacy and 
failure. One misspelling in a letter from a stranger may influence the reader 
of thf letter more than anything else on the page. 



64 



These, then, are the kinds of educational experiences (both intramural and 
o<t>»amural) that we could reasonably assume our post-high school students 
had with spelling before they encountered our CAI course. Since their cur-eit 
•^st>-uctors judged our future students to be still deficient in spelling s< ils, 
n seemed logical that we should prepare a spelling course for CAI that wa.. not 
■'•."aditional in content, organization, or presentation, 

A Linguistic Approach . As the basi? for our course, we chose lingutj; : 
o/idence about American English spelling, rather than the global impress ^o-v, 
tf.at speakers of the language have about its spelling. There were four mi n 
reasons fo" our choosing a linguistic approach to the teaching of spell in.i ■^ 
zhis course; they are listed and explained here. 

Rationale , The available empirical evidence shows that a linguistic 
rtpp^ach is at least as effective as any other method of teaching speiVn.j 
Indeed, judging from the kinds and variety of spelling problems prevalent 
iiTiong our adult students, we predict that a linguistic approach will, in i 

<?hown to be more effective generally than other approaches. This pred c.ion 
r.JST.s on some apparently contradictory characteristics of a linguistic approach: 
'V is both realistic and optimistic; it is both analytical and synthetic; nwi 

-is a structured approach, but one without rigidity. For example, a Im^^'Stic 
ftpp'-oach begins by looking at what spelling really is: an attempt to "ep-.- >ent 
Joeech sounds and words graphically, according to a standard usage. Since 
spelVng is identified as an attempt to represent speech sounds, rather tl„H. as 
maans for doing it, there is an implicit assumption that the attempt ma/ mt 
t*'ways be fu'ly s-iccessful. Beginning with a clear understanding of the na'^^jre 
0- th;: spelling system can help to reduce a student's (and a teacher's') 
hosti'ity and impatience with the txceptions. In other words, if a studern has 
some notion about how spelling irregularities come about, he is less l-kp'\ ':o 
v-ew them as senseless trouble spots, nearly impossible to master. Menu-, if? 
may attack spelling more optimistically than if he were told that it is i^ ^tnble 
of contradictions and inconsistencies occurring at random— but which he inui . 
learn, 

A linguistic approach to spelling is analytical because instead of cor ' on- 
trating on an arbitrary string of letters that forms a certain word, U anr yzes 
the identifying traits and locations of each component in the niven word fnd 
sometimes of the linguistic environment in which the word occurs.) For p^ir-f-le. 
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we can linquistically classify a letter acco»^dina to whether it is a vowel or a 
consonant; whether in a given word, it is voiced, voiceless, or silent; whether 
it has an initial, medial, final position in its syllable; whether its syl- 
lable is accented or not; whecher its syllable, if accented receives primary 
stress or iss\ whether it is preceded o» followed by one or more vowels or 
by one or more consonants; the number of syllables in the v/ord and their 
locations in relation to the syllable in which the letter of interest occurs; 
and which other letters occur in the word, as well as their auditory and 
spatial relations to the letter under scrutiny in the given word. Although the 
preceding may appear to verge on taxonomic farce, it isn*t because it does much 
more than merely classify letters and show static relationships between them. 
The componencial analysis of linguistics is a dynamic taxonomy; otherwise it 
would have no relev^^nce for any part of language study. 

Because a linguistic approach to spelling is ordered, it has a discernible 
structure; but because it grows out of languaqe--a changing phenomenon--it also 
has to be flexible. It has to be able to accommodate modifications as the 
student recognizes a need for them For instance, he may recognize stress as 
his crucial clue (i.e., analytical component) to spelling (e.g., the white hous e 
or The White House ); he may recognize that the clue lies in the racial identity 
or the social class of the speaker (e„g., thje man^ or The Man ) ; or in the geo- 
graphic location (e.g. , Stonr damage was heavy ini the Centre County area, or 
Storm damage was heavy in the center county area. ) ; or in the presence or 
absence of a pause (e.g., Tuesday , weld, or Tuesday Weld ). 

The more the student learns to identify significant differences, the more 
he benefits from a linguistic a,oprGdch to spelling. For, by considering letters 
in terms of components like those listed above, students not only observe the 
presence or absence of a given component, but they begin to discern patterns 
and to generalize about necessary and sufficient conditions for particular 
occurrences of letters. Gradually, the analytic practices can lead to empiri- 
cally generated rules, and ultimately to a sentient grasp of spelling. 

Since other approaches have not succeeded in teaching spelling to our 
post-high school students, a linguistic approach i^ not likely to do less than 
the others have done. However, we repeat that a linguistic approach to spelling 
promises much more than other methods: It has a potentirl for bringing order 
out of chaos. 
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behavioral objectives . A linguistic approach to spelling is compatible 
wth -"eputdble educational thought and practice. Today, much emphasis is 
placed on knowing what takes place at the behavioral level in a learning situ- 
ation, rather than on philosophizing about what may be taking place at other, 
usually intangible, levels. That is, learning is planned and evaluated— o is 
supposed to be--with a behavioral frame of reference. This is particulaHv 
f-ue ^^o" an CAI courses. Therefore, when we started planning this course, we 
p<'epared two lists of behavioral objectives because we had two entirely differ- 
ent a'ms for the course. One set includes the objectives we have for ourse ves, 
as authors, exploring the capabilities of the instructional system. Since tnis 
j-et of objectives is a departure from the usual, it is discussed elsewhere. 

The other set of objectives, however, centers on the students, as beun- 
ioral objectives normally do. This set deals with these major topics: enter- 
•*ng behaviors, performance behaviors during and after instruction, and the 
criteria fo"^ those performances. For example, we assumed that our students 
would already be able to: read at an average adult rate; follow directions; 
type single words and symbols; and hold a pencil-like object. Therefore, we 
listed these as entenng behaviors (i.e., prerequisite learnings and skills.. 
Although these entering behaviors may seem obvious, failure to consider them 
precisely cm lead to failure and disappointment, if not to disaster. 

[■valuation criteria . Next, we designate what a student would have to <io 
to show us on a pretest that he could spell sufficiently well to warrant hrs 
not taking our course. We decided that if he could identify the correct spell- 
ing of specified words in eight or more sentences located in 10 separate groups 
of five sentences each, he did not need the course. That is, we planned a 
sentence test covering five common spelling er«"or categories. Each category is 
p-^esented fn 10 different sentences, and each sentence has three spelling 
options The five error categories we chose are these: final consonant 
doubling, i^-ei words, plural formation, final e words, and final i^-to-j[ woris . 
Thus, students who select eight or more correct spellings from 30 options u» 
each of the five categories are excused from taking the course. Students who 
select fewer than eight correct spellings in any single category are judged 
deficient In spelling ability and they receive instruction in as many of the 
five categories as their pretest performances indicate they are deficient <r. 
Precisely then, the behaviors stipulated for our pretest are the recognition 
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and selection of correctly spelled viords. The acceptable (i-e,» criterion) 
performance for those behaviors is set at eight out of 10 correct selections 
in five separate error categories (i.e,, an 80^ accuracy in eacn category). 

For the posttest, w& .listed the same behavioral objectives. However, to 
decrease the possibility* of lucky guessing, v^e decided to double the length 
of the test, covering the same five e^^'^o*^ categories with 20 sentences each. 
Consequently, in order to achieve the 80% criterion performance level, students 
have to identify 16 or more correct spellings from 60 options in each of the 
five categories. Students who do not attain an 80% accuracy in all five cate- 
gories, are judged to stil'i have spelling deficiencies in those error catego- 
ries where they select fewer than 16 correct spellings. 

For the five instructional portions of the course, which parallel the five 
error categories, we varied the behavioral objectives in accord with 1) the 
material being taught, and 2) the way in which we v/ere then using the CRT to 
teach that material. For instance, in nearly every instructional portion, a 
student, in order to answer questions correctly, is required to perform each of 
the following cognitive operations: recognize elements and patterns (e,g,, 
letters, words, shapes); differentiate among elements and/or patterns; identify 
and/or label elements and/or patterns; const»^uct elements and/or patterns; 
relate elements to patterns and vice versa; infer, generalize, and/or formulate 
rules from elements and patterns. 

In selecting these behaviors we were guided by the structure of a linguis- 
tic approach. So it is inevitable that a variety of mental operations would be 
required of our students. This inevitability, coupled with the assumed program- 
ing flexibility of the CRT, seems to make it possible for students to discover 
that spelling has more logic and order than they might have suspected from 
having performed mainly cognitive memory operations in the past. 

In sunmary, just as linguistics focuses on language as it is actually used, 
the behavioral objectives imposed by linguistics and the CRT forced us to focus 
on what was actually happening in our course Both linguistics and behavioral 
objectives provided empirical guidelines and checks for us as we worked; and 
empirical evidence of almost any kind is in acco>^d with educational thought 
today. Thus, a linguistic approach to spelling, almost by definition, has to 
be compatible with reputable educational thought and p>^actice--especially if 
the approach is used in a course built on behavioral objectives. 
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A patterned challenge . A linguistic approach offers older spelling students 
c new perspective on an old problem. The novelty of this approach for students 
e ^customed to learning spelling mainly by rote is self-evident, and we have just 
c! scussed the variety of cognitive ski Us inat the approach entails. But it 
ni ght be appr^opriate to explain why it is especially important to recognize one 
r-i'-non why the novelty of this new and more demanding approach to spell inq might 
b'i pan cularly important for our students.. 

tiy virtue of their being high school graduates who were accepted into 
edj;ac'onaT programs at accredited institutions, our students must be regarded 
c u-^rJerach-evers as far as spelling is concerned. One of the most commonly 
g ven explanations for why students fail to achieve academically at a level 
(.ominens'j-^ate with their intellectual ability is that they simply are not chal- 
lenged by the learning tasks assigned to them. Hence, they usually fall into 
the habit of ignoring those tasks altogether or of doing them in the most 
desultory manner possible. When, as was probably the case with our under- 
c-itie.'e-s, the assigners of the tasks provide little logical framework for the 
tisks and instead, often emphasize only arbitrary bases for them, it is not 
jj'-prismg that students fail to learn to spell. 

.A linguistic approach, on the other hand, because it presents spelling in 
c^Ue a different perspective, and because it inherently demands more of the 
svjdent. involves him intellectually in ways that spelling never did before, 
fj'themore, since he learns to detect and relate patterns himself, his learn- 
iM.5 e.xper-'ences are almost certain to be more satisfying, both intellectually 
end emotionally, than memorization is. The very process of finding significant 
e'ementb and relationships in spelling provides a challenge that few of our 
students a^e likely to associate with spelling. In short, by helping them see 
t.'^at spewing is, for the most part, logical, we challenge them to tease out 
tne rat^ona^es behind apparent irregularities. In addition, this sort of con- 
centration may help students learn the correct irregular spellings more ef- 
fi.'';tiveiy than has studying them in lists or in isolation. 

"he new perceptions fostered by the linguistic approach jsed in this 
cou'-se promise to provide strong bases and procedures for learning spelling. 
And perception has been shown to influence learning in many direct and in- 
d.rect ways Hence, the apparent usefulness of a linguistic approach to 
speVr.ng has important implications for underachieving adult students. 
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CAI-CRT Aspects 

In our spelling course for post-high school students, the CAI-CRT aspects 
represent what we did in trying to understand and capitalize on the relevant 
and available capabilities of both of those technologies--and what we acquired 
through serendipity Ironically, our early thoughts about how to accomplish 
full use of our facilities were influenced by two disparate facts: a) CAI is 
often naively criticized as being little more than a very elaborate gimmick; 
and b) the CRT is a TV tube. What naive critics might be saying when they 
heard that the 1500 instructional system student stations used a TV screen was 
not hard to imagine. And what we imagined was very discouraging until we 
began to see possibilities in what might be called an "If-you-can* t-beat-them,- 
join-them." approach. We began to consider the use of so-called gimmicks and 
certain television techniques as ways for capturing the attention and interest 
of our older students so that we could try to teach them spelling. 

At that time, some of us had seen an earlier CAI installation use lines 
of print, static graphics, and moving graphics effectively and innovati vely. 
But none of us had seen or read of anyone's using (or proposing to use) single 
letters or words as if they were themselves graphics „ Nevertheless, the CRT's 
capacity for showing movement seemed particularly worth investigating--at least 
as an attention-getting technique The characteristics of our students and of 
our familiar subject indicated that a fresh approach would be desirable, so 
without knowing exactly how they would be used, the authors began to experiment 
with display techniques. By combining Coursewriter II command statements (e.g., 
pauses, display texts, display text inserts, etc ) we soon found now to achieve 
attractive (in the literal sense of the word) displays on the screen. Gradually, 
as we worked with the 1510 screen, ideas and techniques began to build on them- 
selves. Modifications seemed almost to suggest themselves as we considered 
techniques along with the subject matter of the five common spellinc error 
topics we were teaching in the course It was not very long before we realized 
that the novel effects we had started out using as interest-capturing gimmicks 
had very real advantages for teaching spelling Therefore, we started exami- 
ning our course material in terms of the display techniques we had already 
developed, and asking ourselves what other techniques we could devise that 
would effectively present particular points that we wanted to make in our 
linguistic approach to our subject 
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Some of the techniques that we developed are listed below together with 
their purposes and the effects they produced. Since the linguistic approach 
to spelling emphasizes componential analysis and pattern recognition, display 
techniques that highlight the components or patterns under discussion become 
intrinsic to both the text being presented and to the visual presentation 
medium. 

Consequently, although we first anticipated capitalizing on the CRT's 
similarity to TV as an entertainment medium, we now regard that similarity as 
of minimal importance. The special display effects we use in teaching this 
course capitalize on the use of the medium as an instructional medium under the 
control of a computer. 

Among the special display effects and techniques we used are the following: 
1; The presenting of words on the screen in a spell ed-out mode, that is, 
one letter at a time, not only because it is appropriate for a spelling pro- 
gram, bu* because it is an attention-getter. 

2) Ti-e lelling out of a title by moving appropriate letters— one at a 
time— frot. jutside of a screen and accumulating them correctly in the 
middle, an *ie animation of the then-formed words of the title for emphasis 
and as an -ntion-getter. 

3) T vstematic separation or merging of several root words and suffixes 
listed in a cu.umn to show the natures of the two parts, to demonstrate special 
spelling problems, and to identify points of juncture. 

4) The flashing of right answers for emphasis and reinforcement. 

5) The use of the top or bottom half of the multiplication sign (X) for 
an arrowhead without going beyond the 150C system's dictionary. 
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6) The building up of words in a column from the middle outward to 
demonstrate the doubling principle in spelling: 

bb 
a(EF)e 
sj(abbe)r 

tt 
o(tt)e 
hXbtteTr 
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press 



7) The shortening of the command "PRESS SPACE BAR TO CONTINUE" to 
(alternate coded letters) to allow the use of rows 30 and 31 for 



additional course instructions or directions. 

8) The substitution at various times of "TYPE GO TO PROCEED" for "PRESS 
SPACE BAR TO CONTINUE" to increase slightly the student's sense of being mean- 
ingfully Involved in his progress through the instructional part of a course. 

9) The development of an internal quiz segment that allows a student 
multiple responses for one answer with a light pen. Feedback provides not only 
for a statement appropriate to a*" all-correct multiple response, but it lists 
the right and wrong choices in a partially correct multiple response. 

10) The development of a quiz segment that allows a student multiple key- 
board responses for one answer (either letters or words). The feedback pro- 
vides for the showing of the right choices when all are correct. It also 
provides for the showing of right choices and wrong choices (indicated by 
hyphens) when the answer is only partially correct. 

11) The "whiting" (alternate coding) of various letters within a word to 
emphasize patterns or critical areas. 

12) The "whiting" (alternate coding) of key words in an instruction for 
emphasis. 

13) The presentation of instructions on the screen in appropriate "phrase 
fragments" (one phrase under another) to increase ease of reading and compre- 
hension and to create a symmetrically balanced screen image. 

14) The use of animated visual effects to "imitate" certain auditory 
effects . 

15) The development of a short program (including a sub-routine) which 
allows a student, in effect, to animate (that is, to join or separate) certain 
words on the screen by pressing the appropriate key a specified number of times. 

16) The focusing of attention on a particular part of the screen by 
showing an alternate-coded pattern in the area where new material (e.g., a 
different question) will appear next. This technique is used when one part of 
the screen displays the same text throughout several frames, but another part 
of the screen displays a number of different pieces of text. 

17) The presentation of evaluative feedback (yes, no) under individual 
words in a sentence, along with the retention of the positive evaluations 
until the student has answered all of the questions in that sentence correctly. 
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18) The use of relevant punctuation marks as indicators of negative 
evaluation when students persist in making the same error on internal quizzes. 

In addition to these, we have prepared graphemic representations of the 
options (from which the student selects his answers) on the pretest and the 
posttest. These visual representations of sounds use symbols from the standard 
system dictionary as well as some symbols in a special graphics dictionary. 
They also use space significantly by raising and lowering symbols above the 
line of print and by leaving blank spaces within words. The main anticipated 
advantage of this sound representation system, developed by T. V. Barker of 
Lehigh University, is that every English letter in a spelling word has as its 
pronunciation counterpart a symbol that is easily recognizable as the alphabetic 
letter for which it stands. In the absence of audio facilities, this graphemic 
representation was chosen for students who might want to know how the three 
alternative spellings differed in pronunciation. This pronunciation section 
is programed as an optior il branch the student can make on each test item if 
he wishes. Since it has not yet been tested with students, we can only esti- 
mate its usefulness. That estimate is, however, that it will be worth the time 
consumed In coding and inputting on-line the pronunciation symbols. 

In the pretest and posttest sentences, students respond with a light pen. 
This has proved to be a very economical use of system time since it permits 
even CAI novices to be tested on a relatively large number of items in a very 
short time. 

The content of the test sentences, it might be mentioned, was also in- 
fluenced by the age and experience levels of our students. That is, we sought 
to make the test sentences as different from the usual illustrative spelling 
test sentences as we could without distracting students from the purpose of 
the test. That is, we constructed sentences that have a pseudo- literary flavor 
rather than the usual subject-verb-object sentences customarily used by spelling 
teachers. Since the pretest is the students' introduction to our course, it 
seemed important that he realize from the start that the CRT-CAI treatment of 
spelling was not "the same old stuff" simply shown on a TV screen. 
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Some Preliminary Implications 

Although our course has not yet undergone extensive testing. It does not 
seem amiss to us to mention some of the probable Implications that we have 
become especially aware of AS_we worked on this course. 

1) The holistic approach which the CRT-CAI combination of media permits 
is probably a productive one, especially In the early stages of technological 
development. 

2) The integration of the student needs, the subject matter structure, 
and the technological capacities of a CRT-CAI station appears to be particularly 
promising for remedial students and subjects Vf course authors examine how the 
unique qualities and capacities of each of these elements can be used to en- *^ 
hance and/or compensate each other In a learning situation. 
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